SYSTEM AND METHOD FOR GENERATING AND PRESENTING MULTI- 
MODAL APPLICATIONS FROM INTENT-BASED MARKUP SCRIPTS 

ABSTRACT OF THE DISCLOSURE 

Systems and methods for rendering modality-independent scripts (e.g., 
intent-based markup scripts) in a multi-modal environment and, in particular, to a 
multi-modal user interface for an application, whereby the user can interact with the 
application using a plurality of modalities (e.g., speech and GUI). Preferably, the 
multi-modal interface automatically synchronizes I/O events over the plurality of 
modalities presented. In a preferred embodiment, the multi-modal application user 
interface affords a user a choice of speech and/or graphical user interface modalities, in 
which modality-specific details of presentation are automatically determined by the 
system through the use of built-in rules and optional user-supplied preferences, and need 
not be included in the dialog specification. In one aspect, the system provides immediate 
rendering of the modality-independent document in each of the supported modalities. In 
another aspect, the system provides deferred rendering and presentation of intent-based 
scripts to an end user, wherein the system comprises a transcoder for generating a speech 
markup language script (such as a VoiceXML document) from the modality-independent 
script and rendered (via, e.g., VoiceXML browser) at a later time. Both immediate and 
deferred renderings may be performed in either speech-only or multi-modal fashion. In 
other aspects of the present invention, mechanisms are provided for automatic generation 
of application-specific "help" information in the form of a description of the hierarchy 
and navigational flow of the application, analyzing the manner in which navigation 
through the modality-independent script may be performed, scripted emulation of 
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point-and-click navigation applied to the speech interface, automatic insertion of program 
calls into the voice script invoking functions that will synchronize events occuring at the 
field level between the speech/aural interface and the GUI interface, and automatic 
insertion of program calls into the voice script invoking user supplied functions to 
perform variable initialization and other application-specific tasks. In another aspect, the 
present invention comprises a mechanism for enabling flexible user navigation, help, 
contextual help, and feedback suitable to each presentation modality during the dialog 
interaction. The method of user navigation comprises both hierarchical and 
nonhierarchical traversal options. The system synchronizes the states of each 
presentation modality during the dialog interaction. 
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